ECE 566 : Information Theory - Dr . Thinh Nguyen Lecture 9 - February 2 , 2016

As discussed in previous lecture, a process that is both Markov and time independent is called a stationary Markov Process. A finite state stationary Markov process {Xi} can be viewed as a Markov chain. Fig. 1 shows a two-state Markov chain diagram. In this example, there are two states labeled as a and b. Thus, the values of Xi ∈ [a, b]. Associated with a Markov chain is a transition probability matrix T . Since there are two states, the transition probability matrix T is a 2× 2 matrix whose entry Ti,j are the probability of the chain going from state i in the next time step that it is currently in state j. Specifically, in this example, we have: